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The majority offunding for Extension comes from local, state, and federal dollars; therefore the primary> 
driver for evaluation is accountability for public funds. Evaluation has always been a part of Extension 
program implementation; however, these efforts have historically been considered a necessary 
component rather than a priority. The need for Extension to demonstrate public value is increasing due 
to county and state budget cuts. The ability to provide credible information depends primarily on the 
evaluation activities of Extension professionals. The purpose of this research was to use an 
organizational framework to examine how organizational evaluation structures influenced evaluation 
behaviors of Extension professionals. A survey was used to collect data from Extension professionals in 
eight state Extension systems to examine how their perceptions of specific organizational and individual 
evaluation factors influenced their engagement in evaluation behaviors. The results show changes at 
multiple levels within an Extension system can be used to predict behavior. Extension leaders can impact 
the level at which programs are evaluated by making changes to their own behavior and establishing a 
social culture that is more supportive of evaluation. In addition, an emphasis on evaluation skill 
development for Extension professionals is needed. 
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Introduction 

Existing in some capacity in every state and 
national territory, Extension is a large, 
educationally-focused organization based within 
the land-grant university system. Having 
developed and grown in educational capacity 
over the past hundred years, Extension offers 
some unique challenges when addressing 
evaluation concerns. Nongovernmental funds 
including grants from public and private 
agencies, such as the W. K. Kellogg Foundation, 
assist in the development and delivery of unique 
programs within specific state systems; 
however, the majority of funding for Extension 
programs comes from local, state, and federal 
dollars (Rasmussen, 1989). Therefore, the 
primary driver for program evaluation within 
Extension is accountability for public funds. 

While the federal government has rapidly 
increased Extension accountability requirements 
through passed legislation including the 
Government Performance and Results Act 
(GPRA) in 1993 and the Agricultural Research, 


Extension, and Education Reform Act 
(AREERA) in 1998, Extension continues to 
exist with little data showing programmatic 
worth. This may be attributed to a broadly held 
belief that evaluations are never used by 
decision makers. Patton (2008) argued the 
majority of evaluation efforts are conducted in 
vain. Wholey, Scanlon, Duffy, Fukumotu, & 
Vogt (1970) stated that “the recent literature is 
unanimous in announcing the general failure of 
evaluation to affect decision making in a 
significant way” (p. 46). In fact, very little 
evidence shows evaluations have succeeded in 
effecting government planning (Patton, 2008). 
Weiss (1972) stated “a review of evaluation 
experience suggests that evaluation results have 
not exerted significant influence on program 
decisions” (p. 10). 

Other research shows this situation is 
sometimes different when it comes to Extension. 
By placing an emphasis on learning along with 
accountability, embedding in evaluation system- 
wide policy, Extension evaluation efforts can 
make an impact on decision making and enhance 
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support for state-level funding. During the 
1990’s, Ohio State University Extension used a 
proactive approach to working with their 
legislators and decision makers resulting in 
significant growth in appropriations from their 
state government (Jackson & Smith, 1999). 
Their basic philosophy was guided by striving to 
stay accountable through open communication 
with their funders regarding how resources were 
used in the past, how continued and increased 
funding would be used in the future, and openly 
sharing their evaluation plans and results 
exhibiting public value throughout the 
programmatic process (Jackson & Smith, 1999). 
By documenting positive impacts on clients to 
those making funding decisions, this state was 
able to increase their county Extension budgets 
above and beyond the rate of inflation. 

Unfortunately, the amount of effort the state 
of Ohio puts in to its accountability efforts is the 
exception and not the rule (Fetsch & Gebeke, 
1994). Even with the assistance of Extension 
evaluation specialists, supportive evaluation 
cultures within state Extension systems are 
limited (Radhakrishna & Martin, 1999) despite 
research showing evaluation is an essential 
Extension employee competency (Harder, Place, 
& Scheer, 2010). Due to an initial push to 
measure short term changes, the majority of 
Extension professionals utilize posttests given at 
the conclusion of their educational activities to 
assess the level of success obtained (Franz & 
Townson, 2008). While low level reactions and 
some knowledge and skills gained are accounted 
for with this method, long-term outcomes 
recording actual behavior changes along with 
social, economic and environmental impacts of 
Extension programs are lacking. Without 
enhanced evaluation driven environments at the 
organizational level, the state and federal 
Extension systems will continue to be 
inadequate at reporting programmatic successes, 
resulting in a lower perceived public value of 
Extension programs. 

Therefore, questions exist as to how an 
enhanced evaluation driven environment can be 
established. What is the best organizational 
structure within Extension to promote and 
enhance Extension professionals’ evaluation 
behaviors? Does professional development 
training for Extension professionals emphasizing 
evaluation make a difference? What can be done 
to enhance and promote evaluation behaviors 


leading to credible data on the programmatic 
impacts and public value needed to continue 
government funding? Since identifying and 
using evaluation systems to assess program 
impact in Extension is part of the National 
Research Agenda: Agricultural Education and 
Communication, 2007-2010 (Osborne, 2007), a 
study exploring the ways in which Extension 
systems can be adjusted to encourage the 
creation and use of evaluation systems can yield 
valuable data providing direction for future 
practice. 

Theoretical Framework 

A review of the evaluation literature, the 
Burke-Litwin model of organizational 
performance and change (Burke & Litwin, 
1992), and the Theory of Planned Behavior 
(Ajzen, 1985) led to the creation of a conceptual 
model of organizational evaluation (Lamm, 
Israel, & Irani, 2010) used as the theoretical 
framework for this study (Figure 1). The 
conceptual model of organizational evaluation is 
divided into three sections: transformational 
factors, transactional factors, and individual 
performance factors. Similar to the Burke- 
Litwin model, this model suggests changes 
within the transformational factors influencing 
evaluation practices will require the entire 
system to adjust with individuals across the 
organization exhibiting new behaviors (Burke & 
Litwin, 1992). The transactional factors of the 
model represent the everyday transactions 
occurring within an organization that affect 
evaluation. Changes to these factors are 
considered systematic improvements, 
evolutionary rather than revolutionary, and 
specifically selected (Burke, 2008). The 
individual performance factors within the model 
are derived from both the Burke-Litwin model 
and the Theory of Planned Behavior and 
represent the individual’s needs and values, 
skills and abilities, attitude, subjective norm, and 
perceived behavioral control influencing 
individual behavior choices related to evaluation 
(Ajzen, 2006; Burke & Litwin, 1992). Each of 
the transformational, transactional, and 
individual performance factors will be described 
in more detail below. 
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Transformational Factors 


Transactional Factors 


Individual Performance Factors 



Figure 1. Conceptual model of organizational evaluation (Lamm, Israel, & Irani, 2010) 


Transformational Evaluation Factors 

Leadership is the primary driver in the 
conceptual model of organizational evaluation 
(Lamm et al., 2010) having a strong influence on 
evaluation behavior (Burke, 2008). Arguments 
have been made against the impact leadership 
has on organizations from a social perspective 
(see Salancik & Pfeffer, 1977; Zacarro, 2001; 
Bourgeois, 1985), but in this case Extension 
administration is a hierarchical structure with a 
director for each state program making system- 
wide decisions; therefore their choices and 
opinions weigh heavily on the entire 
organization. Preskill and Boyle (2008) 
determined that leaders who are willing to seek 
information when they make decisions, are open 
to feedback from others, and reward employees 
for engaging in evaluation work will have a 
positive influence on increasing evaluation 
activities and the development of longer lasting 
impacts of evaluation within the organization. 


In addition, an organizational culture 
designed to allow Extension professionals to 
share information freely, trust one another, 
encourage asking questions, and take risks was 
more likely to breed successful evaluation 
efforts (Preskill & Boyle, 2008). While studying 
changes in organizational cultures, examining 
over 200 companies including Con Agra, GE, 
SAS, British Airways, and Bankers Trust, 
Kotter, and Heskett (1992) found that without 
rooting change in the culture of the organization 
it was nearly impossible to sustain the change 
over time. Their primary finding was that those 
organizations with the highest performance and 
ability to make changes were the companies that 
embraced an adaptive culture (Kotter & Heskett, 
1992). 

Transactional Evaluation Factors 

An organizational culture seeking out new 
information will establish policies and 
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procedures that reward the practice of evaluation 
(Burke, 2008). Therefore, using evaluation as a 
benchmark in performance reviews and 
promotion and tenure will influence employees 
to engage in evaluation behaviors (Bess, 1998). 
Burke (2008) also found organizational culture 
influenced the climate within the work unit and 
individuals’ needs and values. Both 
organizational culture and work unit climate are 
social context variables within the conceptual 
model of organizational evaluation. If the 
overall organizational atmosphere encourages 
the use of evaluations, work unit climate will 
follow suit with an emphasis on the need for 
evaluation use as part of the social atmosphere 
(Burke, 2008). 

Since evaluations are most effective when 
management clearly communicates why they are 
necessary and how they are going to be used 
(Preskill & Boyle, 2008), management is 
essential to successful organizational evaluation. 
Currently, management of evaluation activities 
within Extension varies from state to state. In 
some cases evaluation activities are coordinated 
by county or regional directors, while other 
states manage evaluation efforts through the use 
of state wide Extension evaluation specialists 
located closer to state administration (Warner, 
Rennekamp, & Nall, 1996). Through clearly 
created and communicated objectives, and tasks 
to accomplish those objectives, management is 
easier to understand and tasks are more easily 
accomplished by employees (Preskill & Boyle, 
2008). 

The evaluation system includes all aspects 
of the policies and procedures put into place to 
encourage, enhance, and assist with the function 
of evaluation within an organization (Burke & 
Litwin, 1992). High quality evaluation efforts 
require materials, personnel, time, and financial 
resources (Arnold, 2006; Volkov & King, 2007). 
The ways in which a state Extension system sets 
up its reporting procedures, the clarity of goals 
for evaluation, whether evaluation weighs on 
performance appraisals or the tenure process, the 
rewards Extension professionals receive for 
evaluating their programs, and financial 
allocations set aside to allow for evaluation 
impact how much Extension professionals 
evaluate their programs (Arnold, 2006).The way 
a systems policies and procedures are structured 
also influenced professional development efforts 
as dictated by the structure of an organization 


(Burke, 2008). In addition, Arnold found the 
opportunity to learn about and increase skills in 
the area of evaluation through professional 
development efforts impacted individual 
Extension professionals’ evaluation skills and 
abilities. 

Along with professional development, 
structure includes defining levels of evaluative 
responsibility, communication regarding 
evaluation practices within the system, and how 
individuals within the organization interact in 
implementing evaluation (Burke & Litwin, 
1992). Evaluation behaviors are strongly 
dictated by how well an organization is 
structured to create, capture, store, and 
disseminate data (Preskill & Boyle, 2008). 

A positive organizational work unit 
evaluation climate is measured by the ways 
Extension professionals choose to talk about 
evaluation, their willingness to interact about 
and ask evaluative types of questions, their 
interest level regarding the use of evaluative 
data, and a group commitment to conducting 
meaningful and timely evaluations (Boyle, 
Lemaire, & Rist, 1999; Huffman, Lawrenz, 
Thomas, & Clarkson, 2006; McDonald, Rogers, 
& Kefford, 2003). The work unit evaluation 
climate is a key factor in establishing subjective 
norms around evaluation within individuals. 
The work unit climate is the primary social 
influence on an individual’s belief structure, so 
how people the individual respects and of whose 
opinion holds value for the individual view 
evaluation (including peers, management, and 
leadership) will weigh heavily on how that same 
individual established their subjective norm of 
evaluation (Ajzen, 1991). In addition, if the 
work unit accepted and incorporated evaluation 
into the established climate, the individual feels 
more support, and therefore more control over 
using evaluation, enhancing their perceived 
control of engaging in evaluation behaviors 
(Ajzen, 2006). 

Individual Performance Factors 

In order to sustain individual evaluation 
behaviors within an organization long term, 
opportunities must be available for employees to 
learn from and about evaluation (Preskill & 
Boyle, 2008). The structure of the organization, 
including professional development efforts, 
directly impacts the individual evaluation skills 
of Extension professionals (Burke, 2008). 
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Extension professionals must feel they have the 
competencies necessary to perform evaluation 
tasks in order to carry them out (Ghere, King, 
Stevahn, & Minnema, 2006). Extension 
professionals are hired for their subject matter 
knowledge (Rasmussen, 1989) and very rarely 
come with the competencies needed to properly 
evaluate. The amount of professional 
development and the skills/competencies 
Extension professionals within a state system 
have developed in regards to evaluation will 
impact their level of evaluation efforts (Arnold, 
2006). 

Beyond this, Extension professionals need to 
feel evaluating their programs holds personal 
and organizational value in order to be 
motivated to engage (Burke, 2008). Past 
research has shown evaluation use positively 
influences behavior regarding developing future 
evaluations because it offers a feeling that the 
work is truly valued and needed (Mackay, 2002; 
McDonald et al., 2003; Patton, 2008). The 
creation of a culture that uses the results of 
evaluations, thereby altering an individual’s 
need to evaluate, can alter a person’s attitude 
towards evaluation (Burke, 2008). 

If an Extension professional feels the value 
of evaluating outweighs the cost and time 
associated with it, they will engage in evaluation 
practices (Ajzen, 2006). However, people 
experience tension when they are confronted 
with information that challenges their attitudes 
(Eagly & Chaiken, 1993). This is due to a 
feeling that their self-concept or personal 
identity is being challenged (Eagly & Chaiken, 
1993). Individuals want to be seen a certain 
way, so the work unit evaluation climate (social 
atmosphere surrounding evaluation the 
individual works in) will have an influence on 
an individual’s attitude toward evaluation 
(Ajzen, 2006). Research has shown that work 
unit evaluation climate, including talk about 
evaluation in the workplace, team members 
asking questions about each other’s work, group 
interest in using data to make decisions, and the 
creation of group commitments to evaluation, 
directly impacted the individuals’ attitudes about 
evaluation in a positive way (Boyle et al., 1999; 
Huffman et al., 2006; McDonald et al., 2003). 

In addition to attitude, previous research has 
shown how powerful the need for conformity is 
on altering human behavior (Asch, 1956; 
Crutchfield, 1955; Sherif, 1935). The more 


difficult or ambiguous a task is, the more people 
rely on one another to develop and interpret the 
task at hand (Deutsch & Gerard, 1955; Kelley & 
Lamb, 1957). In addition, the more attractive 
belonging to a specific group is, the more likely 
an individual is to conform to their expectations 
(Festinger, Gerard, Hymovitch, Kelley, & 
Raven, 1952). An individual’s established 
attitude is a measurement of how much they 
favor or disfavor an activity, therefore the 
attractiveness of a group has a strong reciprocal 
relationship with the subjective norm. It is 
expected that if evaluation practices are a norm 
within the work unit climate the individual is a 
part of and attracted to then they will feel more 
control over the practice, align with what is 
expected, and pursue engaging in evaluation 
behaviors (Ajzen, 2006). 

Purpose and Objectives 

The purpose of this research was to use a 
system-wide approach to examine how 
organizational evaluation structure influences 
the evaluation behaviors of Extension 
professionals in the field. The study was guided 
by the following objectives: 

1. To identify Extension professionals’ 

personal and professional characteristics, 
evaluation behavior, and perceptions 

regarding transformational evaluation 

factors, transactional evaluation factors, and 
individual performance evaluation factors. 

2. To determine if, and how, Extension 

professionals’ perceptions regarding 

transformational evaluation factors, 
transactional evaluation factors, individual 
performance evaluation factors, and 
personal and professional characteristics 
explain variation in Extension professionals’ 
evaluation behavior. 

Methods 

Participants 

The participants for this study consisted of 
the 1795 field-based Extension professionals 
employed by the University of Arizona, 
University of Florida, University of Maine, 
University of Maryland, Montana State 
University, University of Nebraska, North 
Carolina State University, and University of 
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Wisconsin in 2010. Field-based Extension 
professionals were chosen as the target 
population due to their direct programming 
responsibilities, and therefore direct evaluation 
expectations. It is recognized that in some 
states, state Extension specialists conduct 
educational programming similar to field staff, 
and as such the exclusion of these individuals is 
a limitation of this study. Specific state systems 
were targeted to incorporate differences in 
organizational size, location, and hiring process. 
Responses were received from 1223 of the 
participants. Only 1173 were complete, 
resulting in a response rate of 65.2%. 

A Chi-square test was run on the gender 
variable in order to identify if any significant 
differences existed between those who 
responded and the entire sample population 
based on a p value of <.05 established a priori. 
Differences in gender were non-significant with 
a Chi-square value of 2.00 and a p value of .16. 
A Chi-square test was also run on the gender 
variable to determine if any significant 
differences existed between respondents and 
non-respondents. Differences in gender were 
non-significant with a Chi-square value of 1.72 
and a p value of .19. 

Instrumentation 

Due to a lack of previous research on the 
topic, an instrument measuring the variables of 
interest was not available, therefore the 
researcher created a 110 item instrument. The 
conceptual model previously described was used 
as the basis for specifying items for the 
instrument. 

Participants were asked to report on how 
they evaluated their best or most important 
program by marking whether or not they had 
engaged in 29 specific data collection or data 
analysis methods during the past year. To assess 
participants’ perceptions regarding the level 


their organization addresses transformational 
and transactional evaluation factors they were 
asked to rate their perception of specific items 
related to each factor on a Likert-type scale. 
The scale ranged from 1 - Strongly Disagree, 2 

- Disagree, 3 - Neutral, 4 - Agree, 5 - Strongly 
Agree. The survey included questions regarding 
evaluation leadership, organizational evaluation 
culture, the management of evaluation, 
evaluation policies and procedures, work unit 
climate regarding evaluation, and structure 
pertaining to evaluation. 

To assess each participant’s individual 
performance evaluation factors, several types of 
questions were used (Ajzen, 2006). On 
questions related to individual needs and values 
regarding evaluation and individual evaluation 
skills and abilities the instrument requested 
participants rate their perception on a Likert- 
type scale. The scale ranged from 1 - Not at all 
true for me, 2 - Slightly true for me, 3 - 
Somewhat true for me, 4 - Mostly true for me, 5 

- Completely true for me. Participants were 
asked to rate their perceptions of items 
associated with their attitude, subjective norm, 
and perceived behavioral control of evaluation 
on a semantic differential scale (Ajzen, 2006). 
This set of questions asked participants to rate 
their level of agreement between a set of bipolar 
adjectives. The instrument ended with a series 
of questions related to the individual’s personal 
and professional characteristics. 

The instrument was reviewed by a panel of 
experts specializing in survey design, evaluation, 
and familiarity with Extension organizations for 
internal validity and pilot tested on the 
Extension professionals employed by Colorado 
State University to check reliability. All of the 
final indexes created were considered acceptable 
with Cronbach’s alpha coefficients ranging from 
.70 to .86 (Table 1). 
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Table 1 


Number of Items in each Index and Final Index Reliability 


Index 

Number of Items 
in Index 

Reliability 

Coefficient 

Evaluation Leadership 

6 

.85 

Organizational Evaluation Culture 

5 

.79 

Management of Evaluation 

6 

.85 

Evaluation Policies and Procedures 

6 

.70 

Work Unit Evaluation Climate 

5 

.85 

Structure Pertaining to Evaluation 

5 

.81 

Individual Needs and Values Regarding Evaluation 

6 

.82 

Individual Evaluation Skills/Abilities 

11 

.84 

Attitude towards Evaluation 

7 

.86 

Subjective Norm of Evaluation 

6 

.79 

Perceived Behavioral Control of Evaluation 

11 

.80 


Procedure 

The instrumentation for the study was 
distributed using an online survey. The target 
population’s access to the Internet and use of e- 
mail as a communication tool enabled the use of 
an online survey instrument (Dillman, Smyth, & 
Christian, 2009). The researcher designed and 
implemented the online survey by contacting the 
participants via e-mail using Dillman et al.’s 
(2009) Tailored Design Method including an 
initial request to complete the survey and weekly 
reminders for four weeks. A list of Extension 
professionals’ names, e-mail addresses, and 
gender were generated by each state system. 
The state lists were combined, with state coded 
by the researcher to create a database of 1795 
Extension professionals. A participant number 
was assigned to each individual. All 
correspondence related to the actual survey were 
sent by the researcher with approval from each 
state’s Extension administration. Each state 
Extension director sent out an e-mail system- 
wide two weeks prior to the initial contact 
alerting the Extension professionals in their 
system they would be receiving an important 
survey. 

Data Analysis 

Quantitative research methods were used to 
achieve the research objectives. Data analysis 
for the study was completed using SPSS 18 
statistical software package for Windows. 
Descriptive statistics were used to accomplish 
the first objective. There were two stages to 
calculating the level of evaluation behavior 
dependent variable. First, the responses to the 


three program participation record items (keep 
records, track gender, track race/ethnicity) were 
su mm ed. The sum score of these three items 
were multiplied by the response to the 
participation accuracy response (0 - not at all 
accurate, .25 - slightly accurate, .5 - somewhat 
accurate, .75 - mostly accurate, and 1 - 
completely accurate ) to create a participation 
records score. Second, responses to the 26 
remaining behavior items were summed to 
create an evaluation item score. The 
participation records score and the evaluation 
item score were then summed to create a score 
(with a possible range of 0 to 29) used as the 
evaluation behavior score in further data 
analysis. 

Participant responses to the set of statements 
designed to measure each transformational, 
transactional, and individual performance 
evaluation factor were summed and averaged to 
create an overall score for each factor. 
Consistency of measurement within each factor 
was determined by using Cronbach’s alpha 
coefficient. All of the final indexes created were 
considered acceptable, ranging from .70 to .86. 
Frequencies were reported for responses to each 
of the personal and professional characteristic 
items. Multiple regression analysis with 
hierarchical order of entry of predictor variables 
was performed to accomplish the second 
objective. 
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Results 

Objective 1 

Descriptive analysis of the personal and 
professional characteristic data collected showed 
there were 751 female (64.0%) and 422 male 
(36.0%) participants. The large majority 
(87.6%, n = 1027) of participants were 
Caucasian/White with African Americans 
representing 4.1% (n = 48). Hispanic, Native 
American, and Other categories were 
represented minimally. The majority of 
respondents (70.1%, n = 822) had obtained a 
Master’s degree while 19.0% (n = 223) had a 
Bachelor’s degree. All programmatic areas were 
represented with 27.1% (n = 318) of respondents 
focusing on Family and Consumer 
Sciences/Nutrition, 23.4% (n = 275) on 4-H 
Youth Development, 24.6% (n = 289) on 
Agriculture, and 11.2% (n = 131) on 

Horticulture. Almost half of the participants 
(43.1%, n = 505) were in tenure-tracked 
positions with 62.6% (n = 316) having already 
achieved tenure. 

When the participants’ evaluation behaviors 
were reviewed, it was discovered that 13.6% (n 
= 163) did not engage in the practice of 
evaluation. The majority of participants who did 
evaluate kept program participation records (/7 = 


966, 82.4%), tracked their participants’ gender 
(;n = 841, 71.7%), used posttests to evaluate 
specific activities (n = 830, 70.8%), tracked their 
participants’ race/ethnicity (/7 = 805, 68.6%), 
and conducted interviews to evaluate their 
activities (n = 760, 64.8%). Examining data 
analysis/reporting methods revealed the majority 
of participants who evaluated were just reporting 
the number of customers attending their program 
(n = 966, 82.4%). When calculated, the overall 
evaluation behavior scores of the participants 
ranged from 1.00 to 27.25 (M = 11.83, SD = 
6.20) and were normally distributed. 

The transformational, transactional, and 
individual performance factors expected to 
influence evaluation behavior were also 
examined (Table 2). Of the individual 
performance factors, participants had the highest 
level of agreement with a perceived subjective 
norm around evaluation (M = 4.16, SD = .67). 
In addition, participants reported a higher level 
of agreement towards needing and valuing the 
practice of evaluation (M = 3.96, SD = .69). 
There were stronger levels of agreement on 
almost all of the individual performance 
evaluation factors than factors at the 
transformational and transactional levels within 
their organization. 


Table 2 


Participants ’ Perceptions Regarding Transformational, Transactional, and Individual Performance 
Evaluation Factors 


Evaluation Factor 

M 

SD 

Transformational Factors 

Leadership 

3.58 

.65 

Culture 

3.53 

.69 

Transactional Factors 

Structure 

3.73 

.74 

Management 

3.56 

.77 

Policies and Procedures 

3.42 

.63 

Work Unit Climate 

3.40 

.78 

Individual Performance Factors 

Subjective Norm 

4.16 

.67 

Needs and Values 

3.96 

.69 

Attitude 

3.90 

.67 

Skills and Abilities 

3.63 

.59 

Perceived Behavioral Control 

3.60 

.62 


Objective 2 

Initially both transformative evaluation 
factors, leadership and culture, had a significant, 


yet small, influence on the participants’ 
evaluation behavior score variation (see Model 1 
column in Table 3). However, when mediating 
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factors were introduced in Models 2-4, the 
influence of the transformative evaluation 
factors became non-significant. When 
transactional evaluation factors were added to 
the model, structure and work unit climate 
emerged as having a significant positive 
influence on evaluation behavior (see Model 2 
column in Table 3). Upon the addition of the 
individual performance factors, both structure 
and work unit climate became non-significant 
and management emerged as having a 
significant negative effect on evaluation 
behavior. In this model, skills and abilities had a 
large positive significant effect (4.42) along with 
the subjective norm having a large positive 
significant effect (1.29). Changes occurred 


again in the final model, where personal and 
professional characteristics were added (see 
Model 4 column in Table 3). In the last model, 
the only transformational or transactional 
variable having a significant effect was work 
unit climate (-.11). Individual skills and abilities 
and subjective norm remained significant in this 
model showing the personal and professional 
characteristics did not mediate their effect on 
evaluation behavior. In fact, the only personal 
and professional characteristic having a 
significant effect on evaluation behavior was an 
individual’s tenure status. If a participant was 
accruing tenure, their level of evaluation 
behavior score was 1.06 points higher than a 
participant in a non-tenure tracked position. 


Table 3 



Model 1 

Model 2 

Model 3 

Model 4 a 

Transformational Factors 





Leadership 

.72* 

.24 

.22 

.11 

Culture 

Transactional Factors 


-.14 

-.28 

-.24 

Structure 


1 17** 

.11 

.12 

Management 


-.44 

-.58* 

-.50 

Work Unit Climate 


.63* 

-.12 

-.11* 

Policies and Procedures 

Individual Performance Factors 


.38 

.17 

.20 

Needs and Values 



.30 

.18 

Skills and Abilities 



4.42** 

4.51** 

Attitude 



-.47 

-.48 

Subjective Norm 



1.29** 

1.27** 

Perceived Behavioral Control 
Personal Characteristics 



.27 

.31 

Non Tenure Tracked 




— 

Accruing Tenure 




1.06* 

Achieved Tenure 




.46 


Note. a Model 4 also included gender, race/ethnicity, education level, and programmatic area variables 
which were non-significant. * p <.05, **p < .01. 


A very small amount of the variation ( R 2 = 
.02) in participants’ evaluation behavior scores 
was explained by the first model that only 
included the organizational transformational 
factors (Table 4). The addition of transactional 
factors made a small significant contribution to 
the explanation of variance (R 2 Change = .02). 
However, the inclusion of individual 
performance factors made a much larger 


significant contribution to the explanation of 
variance (R 2 Change = .17). The third model 
explained 21% of the variance in Extension 
professionals’ evaluation behavior scores. 
When the personal characteristics were added to 
create the last model, the change in explained 
variance was not significant at an alpha level of 
.05. 
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Hierarchical Regression of Evaluation Behavior on Selected Factors and Characteristics 


Variable Entered 

Adjusted 

R 2 

R' 

Change 

F Change 

Sig. of 
Change 

Transformational Factors 

.02 

.02 

13.11 

.00 

Transformational & Transactional Factors 

.04 

.02 

6.59 

.00 

Transformational, Transactional, & Individual 

Factors 

.21 

Al 

50.61 

.00 

Transformational, Transactional, Individual Factors, 

& Personal Characteristics 

.22 

.01 

1.18 

.29 


Conclusions 

This national study gives insight into the 
evaluation behaviors Extension professionals are 
engaging in. It also sheds light on how 
Extension professionals perceive organizational 
and personal factors associated with evaluation 
and how those factors are contributing to 
variation in their evaluation behaviors. This 
study has shown Extension professionals are 
engaged in a wide variety of evaluation 
behaviors. However, the majority are only 
fulfilling basic reporting requirements. This 
data supports Franz and Townson’s (2008) claim 
that the majority of Extension professionals 
currently utilize posttests given at the conclusion 
of their educational activities to assess the level 
of success. In general, Extension professionals 
reported some agreement with the view that their 
organization addresses evaluation at all levels. 
It is interesting to note Extension professionals 
reported a higher level of agreement with the 
individual performance evaluation factors than 
evaluation factors at the organizational level 
although the mean differences were not great. 
While Extension professionals are not being 
hired for their evaluation expertise, many do 
need and value evaluation and associate 
themselves as having some evaluation skills and 
abilities. 

In addition, the results from this study are 
aligned with previous research on organizational 
change theory showing transformational, 
transactional, and individual performance factors 
have an effect on behavior choices (Burke, 
2008). However, the findings from this study 
are incongruent with previous research on 
individual behavior change theory showing 
attitude, subjective norm, and perceived 
behavioral control directly impact behavior 


(Ajzen, 2006). In this study, only the subjective 
norm had a significant effect on behavior. 

At the transformational level, leadership and 
culture had a significant effect on evaluation 
behavior when other factors were not controlled. 
This result support previous research suggesting 
leaders open to discussions surrounding the 
practice of evaluation, which encourage 
professional development in this area, and use 
evaluation data when making decisions will 
have a positive influence on increasing 
evaluation activities (Preskill & Boyle, 2008). 
The positive effects of these transformational 
factors were mediated by transactional factors. 
Structure and work unit climate were the two 
transactional factors which appeared as 
fundamental to influencing evaluation behavior. 
This is consistent with previous research 
showing that evaluation behavior is enhanced 
when Extension professionals understand their 
evaluative responsibilities, perceive open 
communication within the system regarding the 
practice of evaluation, and are encouraged to 
interact while implementing evaluations (Boyle 
et al., 1999; Burke & Litwin, 1992; Huffman et 
al, 2006). 

The most influential factors were at the 
individual level. In this study an individual’s 
evaluation skills and abilities had the strongest 
influence, indicating that the amount of 
professional development an Extension 
professional receives in regards to evaluation 
has a large impact on their level of evaluation 
efforts (Arnold, 2006). In addition, the 
subjective norm contributed significantly to 
variation in evaluation behavior. This supports 
the view that Extension professionals are relying 
on one another to make decisions regarding 
evaluation behaviors (Deutsch & Gerard, 1955; 
Kelley & Lamb, 1957). 
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Implications and Recommendations 

Extension leaders need to recognize how 
their actions will have effects throughout their 
Extension system, albeit these are relatively 
weak and mediated by other factors. Through 
intentional action emphasizing the importance of 
evaluation, these individuals can have a system- 
wide impact, literally transforming their 
organization to one that values evaluation. This 
intentional action should include the creation of 
a statewide reporting system for creating, 
storing, and disseminating evaluation data 
(Preskill & Boyle, 2008). Extension leaders 
should also work toward shifting the culture of 
their organization to be more pro-evaluation and 
add structures that support the development of 
evaluation skills and abilities. New structures 
might include creating an evaluation focused 
website displaying the answers to common 
questions regarding evaluation issues and 
contact information for peers and specialists 
who understand and are willing to share their 
evaluation expertise. Individuals need to feel 
supported by their peers in order to see value in 
evaluating and feel confident in their abilities as 
shown by the influence the subjective noim 
around evaluation had on evaluation behavior. 
By placing an emphasis on establishing 
networks Extension professionals can turn to 
when faced with difficult decisions regarding 
evaluation, they will be more likely to engage in 
evaluation behaviors (Ajzen, 1991). 


Most importantly, at the individual level, 
Extension professionals must have the skills and 
abilities to evaluate their programs in order to 
engage in evaluation. Since Extension 
professionals are hired for their subject matter 
knowledge (Rasmussen, 1989), evaluation 
focused professional development opportunities 
are essential to gaining these skills. Increased 
financial and human resources should be 
allocated to the development of evaluation skills 
including training on defining measureable 
objectives, creating logic models, working with 
stakeholders to establish evaluation needs, data 
analysis, reporting results, and communicating 
results effectively (Ghere et al., 2006). With a 
strong emphasis placed on training, so that 
Extension professionals have the skills necessary 
to evaluate and report findings, state and federal 
Extension systems will be more likely to obtain 
data showing programmatic worth. 

This study shows changes at all levels 
within an Extension system will have an effect 
on Extension professionals’ evaluation 
behaviors. By making major changes at all 
levels of the organization regarding the practice 
of evaluation, the perceived public value of 
Extension programs can be strengthened as the 
ability to report programmatic success improves. 
With increasingly limited resources available for 
government programs, the federal Extension 
system will be better positioned to justify the 
continuance of the funds it receives. 
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